The  Department  Of  Defense  Explosives  Safety  Board 
Explosives  Safety  Management  Program  Evaluation  Process 


Eric  Alchowiak 

Director,  Program  Evaluation  Division 

DoD  Explosives  Safety  Board 
2461  Eisenhower  Avenue 
Hoffman  1,  Rm  856C 
Alexandria,  VA  22331-0600 
(703)  325-0892  (221  DSN) 


34th  DDESB  Explosives  Safety  Seminar 
Portland  Marriott  Downtown  Waterfront 
Portland,  Oregon 
13-15  July  2010 


Report  Documentation  Page 


Form  Approved 
OMB  No.  0704-0188 


Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 
VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 


1.  REPORT  DATE 

JUL  2010 


2.  REPORT  TYPE 

N/A 


4.  TITLE  AND  SUBTITLE 

The  Department  of  Defense  Explosives  Safety  Board  (DDESB)  Explosives 
Safety  Management  Program  Evaluation  Process 


6.  AUTHOR(S) 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS (ES) 

Program  Evaluation  Division  DoD  Explosives  Safety  Board  2461 
Eisenhower  Avenue  Hoffman  1,  Rm  856C  Alexandria,  VA  22331-0600 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS  (ES) 


3.  DATES  COVERED 


5a.  CONTRACT  NUMBER 


5b.  GRANT  NUMBER 


5c.  PROGRAM  ELEMENT  NUMBER 


5d.  PROJECT  NUMBER 


5e.  TASK  NUMBER 


5f.  WORK  UNIT  NUMBER 


8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 


10.  SPONSOR/MONITOR’S  ACRONYM(S) 

11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 


12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release,  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

See  also  ADM002313.  Department  of  Defense  Explosives  Safety  Board  Seminar  (34th)  held  in  Portland, 
Oregon  on  13-15  July  2010,  The  original  document  contains  color  images. 

14.  ABSTRACT 

This  paper  describes,  explains,  and  updates  the  status  of  the  Department  of  Defense  Explosives  Safety 
Board’s  (DDESB’s)  Explosives  Safety  Management  Program  (ESMP)  evaluation  process.  The  DDESB 
Staff  completed  the  first  ESMP  evaluation  year  in  FY2009  and  began  their  second  evaluation  year.  As  part 
of  the  evaluation  process,  the  DDESB  Staff  collected  lessons  learned  and  comments  from  the  field  on  the 
Staff’s  implementation  of  the  process.  Using  this  information,  the  Staff  continuously  assessed  and 
improved  the  process.  Additionally,  DDESB  Staff  observed  that  many  explosives  safety  personnel  and  their 
leadership  do  not  fully  understand  the  new  methodology,  which  emphasizes  identification  of  systemic 
problems  and  emerging  issues  for  improving  the  Service’s  and  DoD’s  ESMP.  To  increase  understanding  of 
the  ESMP  process,  this  paper  presents  and  discusses  DDESB  Staff’s  improvements  and  evaluation  process 
mythology. 


15.  SUBJECT  TERMS 


16.  SECURITY  CLASSIFICATION  OF: 


a.  REPORT 

unclassified 


b.  ABSTRACT 

unclassified 


c.  THIS  PAGE 

unclassified 


17.  LIMITATION  OF 

18.  NUMBER 

ABSTRACT 

OF  PAGES 

SAR 

36 

19a.  NAME  OF 
RESPONSIBLE  PERSON 


Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


The  Department  of  Defense  Explosives  Safety  Board  (DDESB) 
Explosives  Safety  Management  Program  Evaluation  Process 


ABSTRACT 

Eric  Alchowiak 

Director,  Program  Evaluation  Division 
DoD  Explosives  Safety  Board 
2461  Eisenhower  Avenue 
Hoffman  1,  Rm  856C 
Alexandria,  V A  22331-0600 
(703)  325-0892  (221  DSN) 
Eric.alchowiak@ddesb.osd.mil 


This  paper  describes,  explains,  and  updates  the  status  of  the  Department  of  Defense  Explosives 
Safety  Board's  (DDESB's)  Explosives  Safety  Management  Program  (ESMP)  evaluation  process. 
The  DDESB  Staff  completed  the  first  ESMP  evaluation  year  in  FY2009  and  began  their  second 
evaluation  year.  As  part  of  the  evaluation  process,  the  DDESB  Staff  collected  lessons  learned 
and  comments  from  the  field  on  the  Staffs  implementation  of  the  process.  Using  this 
information,  the  Staff  continuously  assessed  and  improved  the  process.  Additionally,  DDESB 
Staff  observed  that  many  explosives  safety  personnel  and  their  leadership  do  not  fully  understand 
the  new  methodology,  which  emphasizes  identification  of  systemic  problems  and  emerging 
issues  for  improving  the  Service's  and  DoD's  ESMP.  To  increase  understanding  of  the  ESMP 
process,  this  paper  presents  and  discusses  DDESB  Staffs  improvements  and  evaluation  process 
mythology. 


The  DDESB  Explosives  Safety  Management  Program  Evaluation  Process 
Eric  Alchowiak,  34th  DDESB  Explosives  Safety  Seminar 


Explosives  Safety  Management  Program  Evaluation  Process 

The  United  States  Department  of  Defense  (DoD)  Explosives  Safety  Program  originated  as  a 
formal  program  in  the  aftermath  of  World  War  I  when  several  ammunition  storage  areas  were 
destroyed  in  a  series  of  mishaps.  The  most  serious  occurred  at  Lake  Denmark  Naval 
Ammunition  Storage  Depot,  New  Jersey,  in  July,  1926  when  an  electrical  storm  led  to  fires 
causing  explosions  and  widespread  destruction.  The  severe  property  damage  and  19  fatalities  led 
Congress  to  empower  a  board  of  Army  and  Naval  officers  to  investigate  the  Lake  Denmark 
disaster  and  determine  if  similar  conditions  existed  at  other  ammunition  depots. 

The  board  reported  in  its  findings  that  this  mishap  could  recur,  prompting  Congress  to  establish  a 
permanent  board  of  officers  to  develop  explosives  safety  standards  and  ensure  compliance.  This 
was  accomplished  when,  in  1928,  Congress  passed  10  USC  Section  172,  Ammunition  storage 
board. 


10  USC  Section  172.  Ammunition  storage  board 

a.  The  Secretaries  of  the  military  departments,  acting  through  a  joint  board 
selected  by  them  composed  of  officers,  civilian  officers  and  employees  of  the 
Department  of  Defense,  or  both,  shall  keep  informed  on  stored  supplies  of 
ammunition  and  components  thereof  for  use  of  the  Army,  Navy,  Air  Force,  and 
Marine  Corps,  with  particular  regard  to  keeping  those  supplies  properly 
dispersed  and  stored  and  to  preventing  hazardous  conditions  from  arising  to 
endanger  life  and  property  inside  or  outside  of  storage  reservations. 

b.  The  board  shall  confer  with  and  advise  the  Secretaries  of  the  military 
departments  in  carrying  out  the  recommendations  in  House  Document  No.  199 
of  the  Seventieth  Congress  (10  USC  172  -  Ammunition  storage  board). 


Figure  1:  DDESB  Publications 
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This  organization  evolved  into  the  Department  of  Defense  Explosives  Safety  Board  (DDESB). 
Today,  the  DDESB  authors  Directive  6055.9E,  Explosives  Safety  Management  and  the  DoD 
Explosives  Safety  Board  (Directive),  DoD  Instruction  6055.16,  Explosives  Safety  Management 
Program  (Instruction),  DoD  4145.26,  DoD  Contractor’s  Safety  Requirements  for  A&E,  DoD 
6055.9-STD,  Ammunition  and  Explosives  Safety  Standards  (Standard),  and  DoD  4145.26-M, 
DoD  Contractor’s  Safety  Manual  For  Ammunition  and  Explosives  (Contractor  Safety  Manual) 
(see  Figure  1).  These  publications  fulfill  half  of  their  congressionally  mandated  mission  of 
oversight  and  keeping  military  munitions  properly  dispersed. 


Figure  2:  Munitions  Life  Cycle 


DDESB  executes  the  second  part  of  the  Congressional  mandate  through  an  evaluation  program, 
which  forms  DDESB  core  component  for  keeping  “informed  on  stored  supplies  of  ammunition 
and  components  thereof  for  use  of  the  Army,  Navy,  Air  Force,  and  Marine  Corps,  with  particular 
regard  to  keeping”  military  munitions  properly  stored  and  preventing  hazardous  conditions 
arising  to  endanger  health,  life,  property  and  environment  damage  inside  or  outside  of  current 
and  former  DoD  installations  developing,  manufacturing,  transporting,  storing,  using, 
demilitarizing,  and  cleaning  up  military  munitions  (10  USC  172  -  Ammunition  storage  board) 
(see  Figure  2).  In  performing  this  core  component,  the  DDESB ’s  Staff  oversees  the  DoD 
Explosives  Safety  Management  Program  (ESMP)  to  enhance  force  protection  and  asset 
preservation  in  a  way  that  preserves  the  Department’s  vital  warfighting  capabilities.  The  Staff 
accomplishes  this  by: 
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a.  Evaluating  the  overall  effectiveness  and  program  performance  of  the  DoD  Components’ 
ESMP; 

b.  Identifying  explosives  safety  management  issues  requiring  DoD  policy  changes  or 
increased  emphasis; 

c.  Providing  information  to  improve  the  DoD  Components’  ESMP; 

d.  Enabling  the  DDESB  to  be  kept  fully  informed  on  ESMP-related  issues; 

e.  Identifying  ESMP-related  concerns,  issues  for  action,  responsible  entity,  and  lessons 
learned;  and 

f.  Helping  to  continuously  improve  DoD’s  and  each  Component’  ESMPs. 
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Evolution  of  Evaluation  Process 

Prior  to  FY  2008,  DDESB  surveyed  DoD  installations  manufacturing,  storing,  maintaining,  and 
using  ammunition  and  explosives  (A&E)  on  a  rotating  schedule.  DDESB  used  these  surveys  to 
visit  every  installation  and  measure  each  installation’s  compliance  with  the  Standard.  Based  on 
the  DDESB ’s  findings,  DDESB  required  each  installation  having  a  noncompliance  with  the 
Standard  to  develop  and  forward  a  plan  through  their  chain-of-command  to  correct 
noncompliance.  In  1929,  the  then  Department  of  War  (Army)  and  Department  of  the  Navy  had 
only  36  installations.  Today,  DoD  consists  over  900  installations  around  the  world.  Over  time, 
the  scope  of  this  effort  has  increased  dramatically. 

In  2003,  the  DoD  Inspector  General  (DoDIG)  recommended  changes  to  how  DDESB  manages 
DoD’s  ESMP.  During  the  DoDIG’s  review  of  DoD  Explosives  Safety  Program  Oversight,  the 
DoDIG  found  program  weaknesses  relating  to  planning,  oversight,  and  identified  the  need  for 
DoD  to  restructure  itself  to  become  more  effective  in  identifying  and  minimizing  risks  associated 
with  explosives  safety  hazards.  In  response  to  the  DoD  Inspector  General’s  Report,  DDESB 
leadership  developed  a  Strategic  Plan  for  restructuring  the  DDESB  and  formulating  an 
implementing  DoD  directive. 

In  optimizing  the  DoD  Explosives  Safety  Process,  DDESB’s  Strategic  Plan  and  the  Directive 
required  the  DDESB  Board  and  Staff  to  develop  a  DoD  Component  explosives  safety  evaluation 
program  in  coordination  with  the  DoD  explosives  safety  stakeholders  (Strategic  Plan  3.4.2). 
Therefore,  beginning  in  FY  08,  DDESB  moved  from  a  standards-based  installation-focused 
survey  to  a  programmatic  review  of  DoD  Component’s  ESMPs  (see  figure  3).  The  review 
measures  the  effectiveness  of  the  DoD  Component’s  ability  to  support  the  warfighting  mission 
while  implementing  explosives  safety  requirements  consistently  across  their  organization.  By 
focusing  on  the  Service’s  implementation  of  their  ESMP  and  their  associated  business  processes, 
DDESB  can  better  assess  how  a  DoD  Component  integrates  explosives  safety  tasks  into  their 
mission  execution. 


Figure  3:  Multi-Level  Evaluation  Process 
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In  the  past,  the  DDESB  Evaluation  Team  might  only  consist  of  two  members  and  would  be  at  an 
installation  for  only  a  day  or  even  less.  Now,  the  Team  normally  consists  of  4  members  and 
spends  3  to  5  days  at  an  organization  reviewing  the  ESMP’s  documentation  and  field 
implementation.  This  additional  time  allows  the  team  to  more  accurately  evaluate  the 
effectiveness  of  the  organization’s  implementation  of  their  DoD  Component’s  ESMP. 

This  new  programmatic  evaluation  process  provides  several  unique  benefits.  One  such  benefit 
relates  to  the  DDESB  Staff  not  only  evaluating  the  installations’  implementation  of  their  DoD 
Component’s  ESMP,  but  the  Staff,  also,  assessing  the  implementation  of  the  DoD  Component’s 
ESMP  from  the  top-down  to  determine  consistency  across  the  DoD  Component’s  command. 

The  programmatic  evaluation  provides  two  other  additional  unique  benefits.  First,  the  DDESB 
Staff  continually  examines  DoD  Component  ESMP  to  determine  where  current  DoD  explosives 
safety  policy  needs  revision  or  new  DoD  explosives  safety  policy  needs  improvement. 
Additionally,  the  DDESB  Staff  constantly  assesses  their  evaluation  process  looking  for  ways  to 
make  the  process  more  valuable  to  DoD  and  the  DoD  Components.  To  aid  in  this  improvement 
process,  the  DDESB  actively  seeks  comments  from  the  DoD  Component’s  personnel  involved 
with  the  evaluation  as  well  as  the  opinions  of  installation  personnel. 

The  focus  of  this  programmatic  evaluation  process  views  noncompliance  as  symptoms  indicating 
challenges  to,  or  a  weakness  in,  the  installation’s  or  middle-level  command’s  management  of 
their  ESMP.  When  the  same  noncompliance  issues  are  observed  at  multiple  locations,  DDESB 
believes  the  recurrences  indicate  challenges  to  or  weaknesses  in  the  Service’s  ability  to 
implement  an  effective  ESMP  consistently  across  the  Service.  Therefore,  DDESB  expects  the 
DoD  Component  to  develop  and  submit  a  plan  on  how  the  DoD  Component  will  surmount  their 
challenges  or  weakness  through  improved  management  of  their  ESMP. 

The  DDESB  Staff  no  longer  requires  or  desires  correction  plans  from  individual  organizations. 
Nor  does,  the  DDESB  Staff  desire  the  DoD  Component  provide  a  corrective  plan  addressing 
individual  organizations.  Rather,  DDESB  wants  a  corrective  plan  addressing  what  each  DoD 
Component  will  do  to  establish,  implement,  and  maintain  their  ESMP  consistently  across  their 
command. 

The  DDESB  Staff  considers  the  observations  collected  at  all  levels  to  be  sample  data  points. 
After  every  evaluation,  the  DDESB  Evaluation  Team  provides  a  briefing  to  all  the  DDESB  Staff, 
discussing  each  of  the  collected  data  points.  These  briefing  have  been  very  valuable  to  the 
DDESB  Staff  for  two  main  reasons.  First,  the  evaluating  Staff  points  out  possible  specific 
management  areas  needing  evaluation  at  future  locations.  Secondly,  the  briefings  have  often  led 
to  refinements  in  the  evaluation  process  based  on  the  Staffs  comments  and  those  of  the  activity 
being  evaluated. 

Since  the  sample  size  is  small,  these  data  points  lack  any  statistical  merit.  Nevertheless,  the 
DDESB  Staff  tries  to  evaluate  at  least  three  installations  with  similar  missions.  If  DDESB  Staff 
observes  the  same  challenges  and  weaknesses  at  two  of  the  three,  the  DDESB  Staff  considers 
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them  to  represent  possible  challenges  and  weaknesses  in  the  DoD  Component’s  ability  to 
implement  a  consistent  ESMP  across  the  Service. 


In  addition  to  looking  for  weaknesses  and  challenges,  the  DDESB  Staff  looks  for  strengths 
during  the  process  (see  figure  4a).  The  DDESB  Staff  includes  these  observations  in  the 
summary  report,  which  is  written  upon  completion  of  the  evaluations.  This  list  of  strengths  is  not 
all  inclusive  of  a  DoD  Component’s  strengths,  but,  rather  lists  those  strengths  DDESB  observed 
being  implemented. 


DDESB  Staff 
^  Collects  Data 
Points 


Trend 

Data 

Points 


Data 

Points 


Ignored 


DDESB  Staff 
Sorts  Data 
Points  Looking 
For  Trends 


Organization 


Organization 


Organization 


Organization 


Organization 

#3 


Organization 

#3 


Figure  4b:  Use  of  Collected  Data  Points 


At  the  end  of  the  evaluation  period,  the  DDESB  Staff  sits  down,  reviews,  analyzes,  discusses, 
and  decides  which  data  points  may  represent  possible  trends  denoting  inconsistencies  in  the  DoD 
Component’s  ESMP,  eliminating  those  that  do  not.  The  Staff  summaries  these  identified  trends 
then  the  Staff  develops  and  recommends  corrective  actions.  The  Staff  provides  these  trend 
summaries  and  recommendations  to  the  DoD  Component  in  a  report  (see  Figure  4b).  The 
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DDESB  Staff  does  not  judge  the  severity  of  the  identified  weaknesses  or  challenges  and  does  not 
include  any  such  rating  in  the  summary  report. 

During  the  development  year,  the  DDESB  Staff  recognized  that  each  evaluation  year  should  be 
dedicated  to  a  single  DoD  Component.  Therefore,  DDESB  focuses  on  one  DoD  Component 
during  each  FY.  This  process  provides  DDESB  with  a  more  relevant  overview  of  the  DoD 
Component’s  ESMP,  since  all  the  data  points  collected  can  be  more  easily  correlated.  Secondly, 
these  cycles  of  evaluations,  allows  the  DoD  Component  to  implement  management 
improvements  prior  to  their  next  survey  cycle,  and  for  seeking  monies  where  necessary  to 
implement  the  corrective  actions. 


Evaluation  Tools 

During  FY  2008,  the  DDESB  Staff  began  developing  various  tools  to  use  in  evaluating  a  DoD 
Component’s  ESMP  at  the  installation  level.  The  Staff  continually  refined  these  tools  and 
developed  additional  tools  to  complement  those  tools  used  at  the  installation  level. 

The  DDESB  Staff  developed  and  uses  three  different  matrices  depicting  the  program  areas, 
elements  and  sub-elements  that  the  Staff  uses  at  the  installation  level  (see  Figure  5),  intermediate 
headquarters  level  (see  Figure  6),  and  at  the  Service  headquarters  level  (see  Figure  7). 


Management 

Plans,  Policies, 
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Execution, 

Operations 

Execution, 
Operations  Support 

1.1  Organization  & 
Staffing 

2.1  Site  Planning 
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1 .2  Tenants 

2.2  MasterPlanning 
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3.3  Demilitarization  and 
Destruction 

4.3  Facility  Maintenance 

1 .4  Risk  Stewardship 
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Figure  5:  Installation  Level  Matrix 
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Intermediate  HQ  Program  Evaluation  Areas 

Management 

Execution 

Specialized  Areas 

1 .1  Organization  and 
Staffing 

2.1  Inspections, 
Evaluations,  Audits,  or 
Surveillance 

3.1  Host  Nation,  Shared 
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Tenants 

1 .3  Issuances 

2.3  Training 

3.3  Munitions  Response 

1 .4  Risk  Stewardship 

2.4  Site  Planning 

3.4  Hazard  Classification 

1.5  Planning 

3.5  Contracts 

Figure  6:  Intermediate  Headquarters  Level  Matrix 


Service  Headquarters  Program  Evaluation  Areas 

DoD  Directive 
6055.9 

DoD  Instruction 
6055.16 

DoD  Standard 
6055.09 

1 .1  General  Tenets 

2.1  Decision  Making 
Elements 

3.1  Management 

1 .2  Program  Management 

2.2  Implementation, 
Resourcing  and  RDT&E 

3.2  Plans,  Policies, 
Procedures 

1 .3  Munitions  Response 

2.3  Board  Level  Items 

3.3  Execution,  Operations 

1 .4  R&D  Investments 

2.4  Other  Responsibilities 

3.4  Execution  and 
Operations  Support 

1 .5  Risk  Stewardship 

Figure  7:  Service  Headquarters  Level  Matrix 


Each  top  level  matrix  is  divided  into  3  or  4  program  areas  depending  on  the  activity  being 
evaluated  (i.e.,  management,  execution,  DoD  Directive,  etc.).  Each  program  area  is  further 
described  by  four  or  five  program  elements  (i.e.,  numbered  1.1, 1.2,  etc.).  Lastly,  each  of  the 
program  elements  is  associated  with  a  goal  and  sub-elements  describing  how  that  goal  is 
achieved.  For  comparison,  Figure  7a  illustrates  a  typical  program  area  matrix  associated  with  an 
installation  level  management  review  while  Figure  7b  illustrates  the  corresponding  program  area 
management  matrix  associated  with  an  intermediate  headquarters  level  review. 
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Area  1  -  Management 
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Is  the  explosives  safety  organizational 
structure  and  staffing  adequate  to  support 
the  organization's  mission 

Clear,  documented 

organizational 

responsibilities. 

Mission  statement 
includes  explosives 
and  explosive  safety 
responsibility. 

Organizational  line  to 
Commander 

Assigned  Explosives 
Safety  Office/Officer 
or  functional 
equivalent 

Explosives  safety 
organizational 
staffing  adequate 
to  support  mission. 

Permanent 
assignment  or 
collateral  duty. 

Formal  training  or 
OJT. 

1.2  Tenants 

Are  tenants  monitored,  controlled  and 
martagedto  ensure  their  explosives  safety 
posture  meets,  or  is  compatible  with,  the 
Service  and  mission  requirements  of  the 
organization 

Tenants  identified. 

Tenant  mission 

identified.  Tenant 
personnel  identified 
and  controlled. 

Tenant  functions  monitored: 
explosives  operations 
approved  or  permitted,  site 
access,  inspection  or  review 
of  tenant  explosives 
activities 

Tenant  command 
reporting  structure. 
Report  thru  host  or 

autonomous. 

Explosives  safety 
review. 

Tenant  MOU,MOA 

or  other 

agreements  in  place 
for  management  of 
explosives  hazards 

1.3  Contractors 

Are  contractors  monitored,  controlled  and 
managedto  ensure  their  explosives  safety 
posture  meets,  or  is  compatible  with,  the 
Service  and  mission  requirements  of  the 
organization 

Contractors  identified 

and  controlled. 

Contracts  written  to  contain 
provisions  requiring 
compliance  with  DoD 
explosives  Safety  Standards 

Contractors  work  to 
explosives  safety 
levels  no  less  stringent 
than  DoD 

requirements 

Contractors  report 
accidents  and  near- 

misses  IAW  DoD 

Standard  or 

contractor  safety 
manual  as 
appropriate. 

1.4  Risk  Stewardship 

A  risk  management  process  ensures  the 
responsible  use  of  resources  in  identifying, 
evaluating,  managing  (i.e.,  preventing, 
controlling,  mitigating) the  potential 
explosives  and  chemical  agent  safety  risks. 

Waivers,  exemptions 
or  certific  ations  in 
place  to  address 
violations  to  DoD 

standards. 

Are  hazards  tracked, 

monitored  evaluated,  and 
management  controls 
adjusted  as  necessary. 

Is  hazard  control 

information 

transmitted  to 
personnel  (i.e., 
personnel  are  aware 
of  explosives  hazards, 
signage,  farmers,  grass 
cutters,  hunters, 
fishing,  etc.) 

Are  risk  decisions 
made  and  accepted 

at  levels 

comparable  to  the 

hazard. 

Figure  7a:  Installation  Level  Program  Area  Matrix 


Area  1  -  Management 

1.1  Organization  and  Staffing:  Explosives  safety  structure  and  staffing  adequately  supports  the  organization’s 
missions. 

Clear,  documented  organizational  responsibilities  that  include  explosives  safety.  Mission  statement  includes 
safety. 

Organizational  line  to  commander  ensures  effective  communication  of  explosives  safety  issues. 

Staffing  levels  are  sufficient  to  assist  subordinate  organizations. 

Assigned  explosives  safety  responsibility.  Effective  routine  two-way  communications  via  organizational  lines  of 
authority. 

1.2  Resource  Allocation:  Explosives  safety  related  requirements  are  identified,  budgeted,  and  resourced  to 
effectively  execute  the  organization’s  ESMP. 

The  budget  has  sufficient  resources  to  support  the  ESMP. 

There  is  a  process  in  place  between  command  levels  to  identify  resource  requirements  and  shortfalls. 

There  is  a  prioritization  process  in  place  that  considers  safety  and  risk. 

The  Safety  Office  is  aware  of  resources  necessary  for  corrective  actions. 

1.3  Issuances:  Explosives  safety  policies,  regulations,  instructions,  etc,  are  developed,  updated,  maintained,  and 
enforced  IAW  DoD,  Service,  and  HQ  requirements. 

Appropriate  and  effective  explosives  safety  policies  and  guidance  are  issued  to  subordinate  organizations. 

There  is  a  process  in  place  to  update  issuances  to  comply  with  DoD  requirements. 

There  is  a  process  to  review  subordinate  organizations’  explosives  safety  issuances. 

Input  is  actively  solicited  from  subordinate  organizations  to  prepare  explosives  safety  issuances. 

1.4  Risk  Stewardship:  Explosives  safety  risk  acceptance  and  management  processes  effectively  indentify, 
evaluate,  and  manage  explosives  safety  risks. 

Responsibilities  and  authorities  are  consistent  with  DoD  and  DoD  Component's  risk  management  requirements. 

There  is  a  process  that  ensures  documented  informed  decisions  (e.g.  ORM,  SOP,  etc)  are  made  by  appropriate 
authorities. 

There  is  a  documented  process  to  prepare,  review  and  approve  deviations. 

Deviations  are  tracked  and  periodically  reviewed  for  completeness  and  applicability. 

1.5  Planning:  Explosives  safety  tenets  and  requirements  are  integrated  into  strategic,  contingency,  and  short-term 
planning. 

Explosives  safety  goals  and  objectives  are  measurable,  tracked,  and  reassessed  as  needed. 

Explosives  safety  is  integrated  into  planning  and  communicated  to  subordinate  organizations. 

Explosives  safety  is  integrated  into  emergency  response  planning. 

Explosives  safety  is  integrated  into  the  BRAC  decision  package. 

Figure  7b:  Intermediate  Headquarters  Level  Program  Area  Matrix 
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These  matrices  are  not  checklists  per  se  since  the  Staff  is  looking  for  systemic  program 
weaknesses  and  challenges  causing  ineffectiveness,  not  solely  compliance  issues.  This  shift  from 
strict  compliance  to  effectiveness  and  compliance  cannot  be  over  emphasized.  Installations, 
intermediate  commands,  or  Service  headquarters  level  commands  should  not  go  through  the 
matrix  checking  off  the  boxes  believing  that  because  those  elements  exist  in  their  program  they 
have  met  the  DDESB’s  goal. 


Figure  8:  Evaluation  Goal 


The  DDESB  staff  goes  beyond  looking  at  whether  an  organization  has  processes  in  place.  The 
Staff  looks  at  how  the  process  is  working  by  interviewing  various  personnel  involved  in  the 
process  -  another  tool  the  Staff  developed  and  uses.  For  instance,  if  the  installation  has  a  work 
order  program,  the  Staff  will  look  at  the  process,  talk  to  the  individual  who  owns  and  manages 
the  program,  look  at  work  orders,  and  talk  with  customers  who  submit  the  work  orders  (see 
Figure  8).  Using  all  these  factors,  the  Staff  determines  if  the  program  is  effective  and  compliant. 

Additionally,  the  DDESB  Staff  conducts  field  observations  to  assess  how  the  various  processes 
are  implemented  and  conducted  at  the  lowest  level  of  the  DoD  Component.  For  instances,  prior 
to  arriving  at  the  site,  the  Staff  will  review  required  explosives  safety  submissions  (RESS), 
compare  the  approved  RESS  with  the  installation’s  RESSs  then  assesses  their  implementation  in 
the  field.  Additionally,  the  Staff  visits  various  storage  locations  to  judge  the  ability  of  field 
personnel  to  implement  DDESB  and  the  DoD  Component’s  storage,  compatibility,  and 
housekeeping  requirements.  While  in  the  field,  the  Staff  observes  the  state  of  the  lightning 
protection  systems  (LPS),  then  compares  the  field  results  with  inspection  records  for  the  LPS  and 
interviews  personnel  responsible  for  maintaining  the  systems. 
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Notification  and  Briefings 

Prior  to  visiting  an  organization,  the  DDESB  Team  Leader  sends  them  a  memorandum,  often 
through  their  DoD  Component’s  Safety  Center  or  headquarters  organization.  The  memo  provides 
a  list  of  the  DDESB  Teams  members,  support  requirement,  documentation  wanted  before  hand 
and  at  the  activity,  activities  to  visit,  and  personnel  to  interview. 


Installation  Provides  In-Brief 


Installation  mission; 


Organization  chart  and  lines  of  communication  for 
explosives  safety  issues; 

Plans  that  impact  explosives  safety  (e.g.,  BRAC 
actions,  MILCON  and  renovation  projects,  land 
actions,  clean-up); 

Summary  of  explosives  accidents  that  occurred  in  the 
last  five  years;  and 

Any  changes  from  information  provided  prior  to  visit 
regardingthe  explosives operationsoccurring  during 
the  visit. 


Figure  9:  Installation  In-Brief 


Another  new  feature  of  the  process  involves  the  organization  being  evaluated  providing  an  in¬ 
briefing  to  the  DDESB  Evaluation  Team  (see  Figure  9).  This  evaluation  should  address  the 
organization’s  mission,  goals,  and  chain-of-command,  emphasizing  the  explosives  safety 
function.  The  in-briefing  should  speak  to  tenants  with  explosives  safety  missions  and  their 
relationship  with  the  organization’s  explosives  safety  program.  The  in-briefing  should  discuss 
the  installation’s  past  explosives  safety  accidents,  deviations,  future  construction,  and  other 
pertinent  explosives  safety  information.  An  electronic  copy  of  the  briefing  will  be  provided  to 
the  Evaluation  Team. 


As  in  years  past,  DDESB  provides  an  in-brief  and  an  out-brief  to  the  Command  group.  However, 
unlike  past  years,  no  report  is  provided  to  the  installation  nor  does  DDESB  require  the 
installation  to  develop  and  summit  a  corrective  action  plan.  However,  both  the  Army  and  the  Air 
Force  safety  organizations  have  requested  copies  of  the  team’s  observations. 
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Year  of  the  Army 

During  FY  2009,  the  Army  volunteered  to  be  the  first  DoD  Component  to  be  evaluated.  During 
the  year,  DDESB  evaluated  the  effectiveness  of  the  Army’s  ability  to  implement  a  consistent 
ESMP  by  visiting  selected  Army  commands  and  installations.  Figure  10.  In  addition  to  the 
information  collected  during  visits,  the  DDESB  staff  consider  other  data,  such  as  accident 
statistics,  and  the  Service’s  interaction  and  responsiveness  with  and  to  DDESB. 


Organization 

Date 

Office  of  the  Director  of  Army  Safety  and  Army  Headquarters 

13  January  2009 

U.S.  Army  Technical  Center  for  Explosives  Safety  (USATCES) 

26  January  2009 

HQ,  U.S.  Army  Corp  of  Engineers 

4  March  2009 

HQ,  Installations  Management  Command 

20  May  2009 

HQ,  Army  Materiel  Command 

27  May  2009 

HQ,  Army  Test  and  Evaluation  Command 

5  June  2009 

PinecastleJeep  Range  (Formerly  Used  Defense  Site  (FUDS)) 

1-3  April  2009 

Fort  Campbell 

8-12  June  2009 

White  Sands  Missile  Range 

22-26  June  2009 

Holston  Army  Ammunition  Plant 

27-31  July  2009 

Anniston  Chemical  Activity  and  Chemical  Demilitarization 

Facility 

27-31  July  2009 

Ft.  Drum 

17-21  August  2009 

Aberdeen  Proving  Ground  (Army  Research  Laboratory,  Garrison) 

14-17  September  2009 

ArmyGarrison  Livorno 

17-22  September  2009 

ArmyGarrison  Grafenwoehr 

24-28  September  2009 

Figure  10:  Army  Commands  and  Installations  Involved  in  2009  Evaluation 

During  this  evaluation  year,  the  DDESB  Staff  and  Leadership  gained  insight  from  their 
observations  and  from  those  of  the  Army  personnel  allowing  the  Staff  to  improve  the  process. 
This  insight  lead  to  many  improvements  and  lessons-learned. 

The  DDESB  Staff  realized  that  the  most  effective  way  for  them  to  gain  a  cross-sectional  view  of 
a  DoD  Component  was  to  nominate  those  installations  they  wanted  to  visit  as  sampling  locations 
to  the  DoD  Component.  Then,  allow  the  DoD  Component  to  propose  the  primary  and  secondary 
dates  for  the  evaluation  in  coordination  with  the  location,  as  well  as  suggest  alternative  locations 
to  the  DDESB  for  negotiation. 

Another  lesson  learned  involved  the  number  of  Army  activities  to  visit.  During  FY  2009,  the 
DDESB  Staff  visited  8  installations  to  gather  sample  data  points.  Given  the  complexity  and 
diversity  of  the  Army’s  mission  and  number  of  locations,  the  DDESB  Staff  realized  the  number 
should  be  greatly  expanded.  Further,  given  the  small  number  of  Marine  Corps  locations,  DDESB 
Staff  decided  that  six  months  was  all  the  Staff  needed  to  evaluate  the  Marines  Corp 
implementation  of  their  ESMP.  Therefore,  the  DDESB  Staff  would  conduct  the  Marine  Corp 
evaluation  during  the  first  six  months  of  the  FY  and  then  the  Staff  would  evaluate  the  Army  over 
the  next  six  months  plus  the  entire  next  FY. 
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The  DDESB  Staff  recognized  a  need  to  communicate  the  results  of  their  evaluations  during  their 
out-brief  at  the  installations.  Since  the  Staff  does  not  leave  a  report,  the  out-brief  is  the  Staffs 
only  opportunity  to  convey  their  results  to  the  organization’s  staff.  Further,  since  the  purpose  of 
the  evaluation  involves  the  collection  of  data  points  for  the  evaluating  the  effectiveness  of  the 
DoD  Component’s  implementation  of  a  consistent  ESMP  across  the  Service,  the  DDESB  Staff 
wanted  to  emphasize  this  focus.  Therefore,  the  DDESB  staff  adopted  a  three-color  rating  system 
-  green,  yellow,  and  red  -  for  indicating  the  state  of  the  ESMP  matrix  element.  DDESB  Staff 
does  not  apply  the  three-color  rating  system  above  the  installation  level. 

a.  Green:  Indicates  an  organization  has  implemented  the  program  element  and  associated 
processes  in  their  ESMP.  Minor  issues  may  exist  but  none  that  affect  the  overall  performance  of 
the  ESMP  element. 

b.  Yellow:  Indicates  local  problems  or  process  weaknesses  exist.  These  local  weaknesses 
usually  affect  a  small  group  of  people  or  an  individual  but  do  not  present  a  pattern.  The  problems 
can  usually  be  traced  back  to  a  particular  person’s  decision,  demeanor,  or  statements.  Local 
problems  are  best  fixed  at  the  level  of  the  organization  that  the  problem  affects. 

c.  RED:  Indicates  one  of  two  possible  conditions  -  systemic  program  weaknesses,  such  as 
lack  of  a  process,  or  a  recognized  Imminently  Dangerous  condition  to  workers  or  the  public. 

The  DDESB  Staff  noted  during  the  evaluation  cycle  that  more  often  than  not  the  organization 
being  evaluated  did  not  understand  the  focus  of  the  new  DDESB  process.  The  organizations  still 
perceived  the  evaluations  as  compliance  surveys.  While  the  DDESB  Staff  has  a  responsibility  to 
identify  and  point  out  noncompliances  found  at  installations,  the  focus  of  the  evaluation  remains 
the  DoD  Component’s  ability  to  implement  their  ESMP  consistency  across  their  command.  The 
DDESB  Staff  views  these  discoveries  of  non-compliant  as  teaching  opportunities  rather  than 
findings  of  fault.  Further,  to  encourage  installations’  openness,  no  report  is  left  with  the 
installation  as  stated  previously.  Nevertheless,  both  the  Army  and  Air  Force  have  asked  DDESB 
Staff  to  provide  a  listing  of  the  observations  found  at  each  activity.  How  those  listing  are  used  is 
up  to  the  DoD  Component. 
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Area  1  -  Management 

1.1  Organization  and  Staffing:  Explosives  safety  structure  and  staffing  adequately  supports  the  organization’s  missions. 

Clear,  documented  organizational  responsibilities  that  include  explosives  safety.  Mission  statement  includes  safety. 

Organizational  line  to  commander  ensures  effective  communication  of  explosives  safety  issues. 

Staffing  levels  are  sufficient  to  assist  subordinate  organizations. 

Assigned  explosives  safety  responsibility.  Effective  routine  two-way  communications  via  organizational  lines  of 
authority. 

1.2  Resource  Allocation:  Explosives  safety  related  requirements  are  identified,  budgeted,  and  resourced  to  effectively  execute  the 
organization’s  ESMP. 

_ 

The  budget  has  sufficient  resources  to  support  the  ESMP. 

There  is  a  process  in  place  between  command  levels  to  identify  resource  requirements  and  shortfalls. 

There  is  a  prioritization  process  in  place  that  considers  safety  and  risk. 

The  Safety  Office  is  aware  of  resources  necessary  for  corrective  actions. 

1.3  Issuances:  Explosives  safety  policies,  regulations,  instructions,  etc,  are  developed,  updated, maintained,  and  enforced  IAW DoD, 
Service,  and  HQ  requirements. 

Appropriate  and  effective  explosives  safety  policies  and  guidance  are  issued  to  subordinate  organizations. 

There  is  a  process  in  place  to  update  issuances  to  comply  with  DoD  requirements. 

There  is  a  process  to  review  subordinate  organizations’  explosives  safety  issuances. 

Input  is  actively  solicited  from  subordinate  organizations  to  prepare  explosives  safety  issuances. 

1.4  Risk  Stewardship:  Explosives  safety  risk  acceptance  and  management  processes  effectively  indentify,  evaluate,  and  manage 
explosives  safety  risks. 

Responsibilities  and  authonties  are  consistent  with  DoD  and  DoD  Component's  risk  management  requirements. 

There  is  a  process  that  ensures  documented  informed  decisions  (e.g.  ORM,  SOP,  etc)  are  made  by  appropriate 
authorities. 

There  is  a  documented  process  to  prepare,  review  and  approve  deviations. 

Deviations  are  tracked  and  periodically  reviewed  for  completeness  and  applicability. 

1.5  Planning:  Explosives  safety  tenets  and  requirements  are  integrated  into  strategic,  contingency,  and  short-term  planning. 

Explosives  safety  goals  and  objectives  are  measurable,  tracked,  and  reassessed  as  needed. 

Explosives  safety  is  integrated  into  planning  and  communicated  to  subordinate  organizations. 

Explosives  safety  is  integrated  into  emergency  response  planning. 

Explosives  safety  is  integrated  into  the  BRAC  decision  package. 

Figure  11:  Intermediate-Level  Matrix 


During  the  development  of  the  evaluation  process  in  FY  2008,  the  DDESB  Staffs  main  effort 
involved  developing  the  criteria  used  at  the  installation  level.  The  DDESB  Staff  spend  some 
time,  as  well,  developing  criteria  for  the  DoD  Component’s  headquarters  element.  During  the 
Army  evaluation,  the  DDESB  Staff  realized  that  neither  criteria  could  be  used  to  evaluate 
intermediate  commands.  As  a  result,  the  DDESB  Staff  developed  the  intermediate  criteria  shown 
in  Figure  11,  and  is  evaluating  its  effectiveness  this  year  with  the  Air  Force. 

At  the  conclusion  of  the  Army  evaluation,  the  DDESB  Staff  developed  a  process  to  review  all 
the  data  points  collected,  determined  which  represented  possible  weaknesses  and  challenges  to 
the  Army’s  ESMP,  and  then  presented  them  to  the  Army  in  a  report  (see  Figure  12).  To  begin  the 
process,  the  DDESB  Staff  set  aside  one  week  to  review  and  discuss  the  data  points.  During  this 
process,  many  of  the  data  points  were  determined  as  not  indicating  trends  and  were  not  used.  At 
the  end  of  this  period,  the  Staff  had  identified  possible  issues  with  the  Army’s  ESMP. 
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Figure  13:  Report  Format 

The  next  step  was  to  put  the  identified  weaknesses  and  challenges  into  a  report.  Since  this  report 
format  would  be  the  template  used  for  many  years,  DDESB  Staff  went  though  many  revisions 
until  finally  arriving  at  a  final  format  (see  figure  13).  The  final  format  lists  the  strengths  and 
issues  in  table-format  with  the  recommended  corrective  actions.  The  Staff  details  each  issue  in  a 
separate  appendix.  This  format  provides  an  overview  of  each  issue  in  the  main  section  while 
providing  the  additional  information  in  the  back  necessary  for  enumeration  and  clarification. 
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Conclusion  and  Future 

The  programmatic  evaluation  process  produces  a  different  type  of  result  than  past  surveys  as 
well  as  shifts  the  responsibility  for  compliance  inspections  from  the  DDESB  Staff  to  the  DoD 
Component  itself.  In  the  past,  the  surveys  stressed  strict  compliance  with  the  Standard.  When 
non-compliance  issues  were  found,  the  installation  was  required  to  develop  and  submit  a 
corrective  action  plan  through  its  chain-of-commend  to  DDESB.  Now,  DDESB  looks  at  the  how 
effective  the  various  DoD  Components  are  in  implementing  their  ESMPs  consistently  across 
their  commands.  The  DDESB  Staff  believes  this  changes  DDESB’s  focus  to  identifying  the 
systemic  problems  at  the  DoD  component  level  rather  than  identifying  the  symptoms  at  the 
installation  level,  as  in  the  past. 

Based  on  the  two  plus  years  of  experience  with  the  new  evaluation  process,  the  DDESB  Staff 
believes  that  the  process  will  never  be  static,  but  rather  will  evolve  constantly  with  each  passing 
cycle.  The  Staff  tries  not  to  change  the  evaluation  process  during  the  evaluation  year  in  fairness 
to  the  DoD  Component  being  evaluated. 
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Area  1  -Management 

Elements 

Evaluation  Goals 

Evaluation  Sub-Elements 

1.1  Organization  and 
Staffing 

is  the  explosives  safety  organizational 
structure  and  staffing  adequate  to  support 
the  organization's  mission 

Clear,  documented 

organizational 

responsibilities, 

Mission  statement 
includes  explosives 
and  explosive  safety 
responsibility. 

Organizational  lire  to 

Commander 

Assignee  Explosives 
Safety  Office/Officer 
or  functional 
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Explosives  safety 
organizational 
staffing  adequate 
to  support  mission. 
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Formal  training  or 
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1J  Tenants 
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Tenants  identified. 
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identified,  Tenant 
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explosives  operations 
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access, inspection  or  review 
of  tenant  explosives 
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Tenant  command 

reporting  structure. 
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Explosives  safety 
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Tenant  MQU,  MCA 

or  other 

agreements  in  place 
for  management  of 
explosives  hazards 

14  Contractors 

Are  contractors  monitored,  controlled  and 
managed  to  ensure  their  explosives  safety 
posture  meets,  or  is  compatible  with,  the 

5  e  rv  i  ce  a  nd  m  i  s  s  i  on  re  qui  re  m  e  nts  of  t  he 
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Contractors  identified 

and  controlled. 
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Contractors,  work  to 
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appropriate, 

1.4  Risk  Stewardship 
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managementcontrols 
adjusted  as  necessary. 

Is  hazard  control 
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signage,  farmers,  grass 
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comparable  to  the 
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